Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: pytest conversion for test lib and infra #1196

Merged
merged 7 commits into from
Apr 24, 2024
Merged

test: pytest conversion for test lib and infra #1196

merged 7 commits into from
Apr 24, 2024

Conversation

IronCore864
Copy link
Contributor

@IronCore864 IronCore864 commented Apr 23, 2024 •
edited
Loading

Refactor test_lib and test_infra to pytest style.

Major changes:

  • Remove unittest.TestCase from the parent class.
  • Parametrize. Refactored a few test cases with combinations of pytest.mark.parametrize.
  • Use pytest's fixture tmp_path, monkeypatch instead of managing temporary directories ourselves and unittest.mock.patch. tmp_path is used in cases where otherwise a request fixture needs to be passed in anyway, there is some explanation in the self-review down below.

Before Refactoring

$ python -m tox -e unit -- test/test_lib.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_lib.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 34 items
====================================================================== 34 passed in 0.18s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61     51     28      0    11%   42-62, 74-114, 122-129
ops/_private/yaml.py           9      2      0      0    78%   28, 33
ops/charm.py                 643    315    177      2    45%   50-80, 135-136, 147, 154-157, 164, 211, 219, 227, 270, 288, 421, 451-464, 471-479, 486-505, 580-582, 589-592, 603-605, 612-613, 647-648, 655-660, 667-686, 735-737, 744-747, 754-758, 782-783, 790-795, 802-806, 817-819, 824-825, 832, 839-840, 870, 888-889, 894, 901-903, 910-911, 923-924, 929, 936-938, 945-946, 954, 1021-1031, 1167-1168, 1171-1193, 1198, 1203, 1208, 1213, 1218, 1226-1235, 1324-1377, 1383-1396, 1399-1420, 1438-1444, 1465, 1505-1521, 1555-1572, 1594-1597, 1610-1611, 1657-1665, 1672-1678, 1703, 1731-1743, 1763, 1772-1782, 1799-1800, 1804, 1809, 1819-1821
ops/framework.py             685    473    270     10    26%   62->exit, 64->exit, 65->exit, 66->exit, 104-119, 123, 126, 129, 132, 137, 142, 147, 152, 157-172, 190-191, 194, 244-245, 252, 259, 280, 288, 303-311, 318, 326-328, 346-350, 361-364, 391-392, 395-408, 415, 425, 430-437, 463-477, 480-488, 492, 495, 498-500, 508, 515-516, 519, 534, 559, 562, 597-598, 604-656, 670-675, 679, 683-689, 693, 698-704, 709-717, 721-737, 741-754, 758, 779-816, 823-824, 828-849, 859, 876-889, 892-957, 961-963, 981-1001, 1012-1021, 1028-1030, 1033, 1036-1037, 1040, 1044, 1048-1049, 1053-1055, 1062-1066, 1069-1081, 1085, 1089, 1093-1097, 1100-1110, 1114-1116, 1145-1146, 1153, 1160, 1166-1210, 1215-1221, 1225-1227, 1231-1235, 1242-1243, 1246, 1249-1250, 1253-1254, 1257, 1260, 1263-1268, 1277-1278, 1281, 1284-1285, 1288-1289, 1292, 1296-1297, 1301-1302, 1305-1310, 1313-1318, 1321-1326, 1329-1334, 1337-1342, 1351-1352, 1359-1360, 1367-1368, 1371, 1374, 1377, 1388, 1391-1396, 1399-1404, 1407-1412
ops/jujuversion.py            80     57     48      0    26%   40-49, 52-58, 61-67, 75-95, 100-103, 107, 111, 115, 124, 130, 135-142
ops/lib/__init__.py          176     25     77      8    85%   117, 132-134, 142-144, 149-150, 152-153, 158-159, 176-182, 192-196, 222->210, 264->268
ops/log.py                    27     17      4      0    32%   29-30, 38, 55-75
ops/main.py                  262    213     92      0    14%   49-52, 56-62, 75-100, 118-122, 132-143, 147, 152-220, 240-247, 251-264, 272-301, 305-308, 316-317, 328-336, 345, 351-368, 375-376, 406-424, 427-429, 432-439, 442-475, 483-494, 504-511, 515, 519-523, 539-546, 555
ops/model.py                1532   1022    628      6    30%   100->exit, 101->exit, 118-130, 138, 146, 158, 163, 172, 177, 189, 197, 205, 215, 225, 245, 259, 279-285, 300, 305, 312-314, 317->exit, 319->exit, 323-330, 349-354, 357, 384-395, 399-413, 432-436, 439, 465-473, 478-485, 504-517, 520, 544-552, 556-566, 569, 579-584, 594-597, 606-608, 616-619, 633-641, 668, 696, 700, 724-735, 757-759, 796, 800-803, 806, 809, 812, 815, 818, 821, 841-849, 852, 855, 858, 861-871, 880, 883-905, 916-917, 924-938, 942, 945, 948, 958-961, 964, 969-978, 985-989, 1026-1042, 1052-1055, 1066-1069, 1091-1112, 1138-1143, 1148-1154, 1164, 1191-1198, 1201-1206, 1211-1214, 1219-1242, 1262, 1281-1289, 1325, 1338-1341, 1349, 1356, 1369-1373, 1392-1397, 1414-1416, 1432-1434, 1451-1453, 1461-1463, 1508-1536, 1539, 1558-1568, 1573, 1576, 1579, 1582, 1585, 1595-1598, 1605, 1609-1613, 1618-1657, 1667-1697, 1703-1705, 1708, 1713-1719, 1722-1723, 1727, 1730-1733, 1736-1740, 1750, 1753, 1769-1771, 1774-1776, 1779, 1795-1799, 1805, 1825, 1842, 1845, 1870, 1909-1910, 1921-1925, 1940, 1951-1953, 1960-1961, 1965, 1968, 1971, 1974-1984, 1995-1998, 2005, 2015-2018, 2023, 2028-2029, 2034, 2039-2042, 2051, 2072-2073, 2076, 2079, 2107-2112, 2130-2145, 2149, 2153, 2157-2160, 2164-2177, 2181-2184, 2201, 2210, 2218-2220, 2228-2233, 2247-2248, 2256-2261, 2265, 2269, 2289, 2326, 2347, 2399-2432, 2484-2506, 2511-2534, 2556-2570, 2590-2596, 2600-2606, 2610-2616, 2642, 2661, 2683, 2705, 2737-2742, 2771-2774, 2782-2787, 2802, 2813, 2824, 2827, 2830, 2833, 2836, 2847, 2850, 2853, 2856, 2859, 2870, 2873, 2876, 2879, 2882, 2894-2899, 2969-2995, 3016-3028, 3033-3060, 3064, 3067-3069, 3072-3079, 3083-3104, 3108-3127, 3130-3150, 3153-3154, 3161-3175, 3178-3179, 3183-3196, 3205-3230, 3241-3243, 3246-3248, 3251-3261, 3264-3269, 3272-3274, 3277-3278, 3283-3284, 3287, 3290, 3293, 3303-3308, 3312-3313, 3322-3331, 3335-3350, 3354, 3366-3373, 3377, 3384-3402, 3406-3411, 3416-3424, 3432-3445, 3453-3469, 3472-3475, 3478-3481, 3484-3487, 3490-3491, 3494-3495, 3502-3511, 3515-3525, 3528-3535, 3542-3543, 3553-3554, 3559-3560, 3567-3574, 3580-3585, 3610-3618, 3621-3622, 3626-3627, 3630-3634, 3661, 3705
ops/pebble.py               1279    890    431     12    26%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 309-314, 318-323, 330-331, 335, 345, 350-352, 359, 385-386, 389, 392, 412-416, 419, 433-434, 437-449, 452, 491-494, 497-506, 528, 533, 536, 551-556, 561, 572, 591-593, 598, 605, 616, 634-642, 647, 661, 678, 697-706, 711, 726, 748-759, 769, 777, 785, 789-795, 799, 804-808, 830-842, 847, 851-859, 862, 866-871, 880-902, 906-930, 938-946, 949, 953-958, 985-987, 991, 996-1004, 1011, 1022-1047, 1051-1063, 1066, 1070-1075, 1097-1106, 1110-1118, 1121, 1124-1129, 1190-1199, 1204-1208, 1222, 1279-1283, 1288-1296, 1305, 1377-1381, 1452-1466, 1469-1471, 1483-1485, 1488-1520, 1534-1561, 1570-1577, 1582-1588, 1597-1611, 1617-1637, 1644, 1648, 1652-1655, 1659, 1666-1668, 1672, 1676-1705, 1709, 1739-1746, 1751-1756, 1772-1781, 1790-1795, 1804-1832, 1836-1837, 1841-1843, 1847-1849, 1855-1859, 1863-1864, 1868-1870, 1887, 1905, 1926, 1947, 1968, 1974-1990, 2015-2019, 2023-2044, 2048-2063, 2068-2077, 2089-2109, 2113-2114, 2122-2126, 2130, 2134, 2156-2194, 2198-2204, 2237-2256, 2264-2275, 2281-2317, 2338-2348, 2375-2384, 2400-2408, 2430, 2452, 2596-2709, 2712-2717, 2720-2722, 2736-2749, 2767-2773, 2790-2800, 2808-2809, 2842-2852, 2859-2882, 2885-2904, 2907-2925, 2929-2931, 2935, 2938-2940, 2943, 2947, 2951, 2955-2962, 2990-3011, 3015-3057, 3067-3092
ops/storage.py               159     98     30      0    32%   44-47, 59-68, 72-84, 90-104, 108, 112, 129-130, 141-146, 153, 157-164, 168, 173, 190-208, 221, 243, 254-258, 265, 269-271, 275-277, 289-293, 301-307, 315, 355-356, 376-382, 393-396, 406, 413, 416
ops/testing.py              1589   1589    768      0     0%   15-3568
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6514   4752   2553     38    23%

After Refactoring

$ python -m tox -e unit -- test/test_lib.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_lib.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 67 items
================================================================ 65 passed, 2 skipped in 0.22s =================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61     51     28      0    11%   42-62, 74-114, 122-129
ops/_private/yaml.py           9      2      0      0    78%   28, 33
ops/charm.py                 643    315    177      2    45%   50-80, 135-136, 147, 154-157, 164, 211, 219, 227, 270, 288, 421, 451-464, 471-479, 486-505, 580-582, 589-592, 603-605, 612-613, 647-648, 655-660, 667-686, 735-737, 744-747, 754-758, 782-783, 790-795, 802-806, 817-819, 824-825, 832, 839-840, 870, 888-889, 894, 901-903, 910-911, 923-924, 929, 936-938, 945-946, 954, 1021-1031, 1167-1168, 1171-1193, 1198, 1203, 1208, 1213, 1218, 1226-1235, 1324-1377, 1383-1396, 1399-1420, 1438-1444, 1465, 1505-1521, 1555-1572, 1594-1597, 1610-1611, 1657-1665, 1672-1678, 1703, 1731-1743, 1763, 1772-1782, 1799-1800, 1804, 1809, 1819-1821
ops/framework.py             685    473    270     10    26%   62->exit, 64->exit, 65->exit, 66->exit, 104-119, 123, 126, 129, 132, 137, 142, 147, 152, 157-172, 190-191, 194, 244-245, 252, 259, 280, 288, 303-311, 318, 326-328, 346-350, 361-364, 391-392, 395-408, 415, 425, 430-437, 463-477, 480-488, 492, 495, 498-500, 508, 515-516, 519, 534, 559, 562, 597-598, 604-656, 670-675, 679, 683-689, 693, 698-704, 709-717, 721-737, 741-754, 758, 779-816, 823-824, 828-849, 859, 876-889, 892-957, 961-963, 981-1001, 1012-1021, 1028-1030, 1033, 1036-1037, 1040, 1044, 1048-1049, 1053-1055, 1062-1066, 1069-1081, 1085, 1089, 1093-1097, 1100-1110, 1114-1116, 1145-1146, 1153, 1160, 1166-1210, 1215-1221, 1225-1227, 1231-1235, 1242-1243, 1246, 1249-1250, 1253-1254, 1257, 1260, 1263-1268, 1277-1278, 1281, 1284-1285, 1288-1289, 1292, 1296-1297, 1301-1302, 1305-1310, 1313-1318, 1321-1326, 1329-1334, 1337-1342, 1351-1352, 1359-1360, 1367-1368, 1371, 1374, 1377, 1388, 1391-1396, 1399-1404, 1407-1412
ops/jujuversion.py            80     57     48      0    26%   40-49, 52-58, 61-67, 75-95, 100-103, 107, 111, 115, 124, 130, 135-142
ops/lib/__init__.py          176     25     77      8    85%   117, 132-134, 142-144, 149-150, 152-153, 158-159, 176-182, 192-196, 222->210, 264->268
ops/log.py                    27     17      4      0    32%   29-30, 38, 55-75
ops/main.py                  262    213     92      0    14%   49-52, 56-62, 75-100, 118-122, 132-143, 147, 152-220, 240-247, 251-264, 272-301, 305-308, 316-317, 328-336, 345, 351-368, 375-376, 406-424, 427-429, 432-439, 442-475, 483-494, 504-511, 515, 519-523, 539-546, 555
ops/model.py                1532   1022    628      6    30%   100->exit, 101->exit, 118-130, 138, 146, 158, 163, 172, 177, 189, 197, 205, 215, 225, 245, 259, 279-285, 300, 305, 312-314, 317->exit, 319->exit, 323-330, 349-354, 357, 384-395, 399-413, 432-436, 439, 465-473, 478-485, 504-517, 520, 544-552, 556-566, 569, 579-584, 594-597, 606-608, 616-619, 633-641, 668, 696, 700, 724-735, 757-759, 796, 800-803, 806, 809, 812, 815, 818, 821, 841-849, 852, 855, 858, 861-871, 880, 883-905, 916-917, 924-938, 942, 945, 948, 958-961, 964, 969-978, 985-989, 1026-1042, 1052-1055, 1066-1069, 1091-1112, 1138-1143, 1148-1154, 1164, 1191-1198, 1201-1206, 1211-1214, 1219-1242, 1262, 1281-1289, 1325, 1338-1341, 1349, 1356, 1369-1373, 1392-1397, 1414-1416, 1432-1434, 1451-1453, 1461-1463, 1508-1536, 1539, 1558-1568, 1573, 1576, 1579, 1582, 1585, 1595-1598, 1605, 1609-1613, 1618-1657, 1667-1697, 1703-1705, 1708, 1713-1719, 1722-1723, 1727, 1730-1733, 1736-1740, 1750, 1753, 1769-1771, 1774-1776, 1779, 1795-1799, 1805, 1825, 1842, 1845, 1870, 1909-1910, 1921-1925, 1940, 1951-1953, 1960-1961, 1965, 1968, 1971, 1974-1984, 1995-1998, 2005, 2015-2018, 2023, 2028-2029, 2034, 2039-2042, 2051, 2072-2073, 2076, 2079, 2107-2112, 2130-2145, 2149, 2153, 2157-2160, 2164-2177, 2181-2184, 2201, 2210, 2218-2220, 2228-2233, 2247-2248, 2256-2261, 2265, 2269, 2289, 2326, 2347, 2399-2432, 2484-2506, 2511-2534, 2556-2570, 2590-2596, 2600-2606, 2610-2616, 2642, 2661, 2683, 2705, 2737-2742, 2771-2774, 2782-2787, 2802, 2813, 2824, 2827, 2830, 2833, 2836, 2847, 2850, 2853, 2856, 2859, 2870, 2873, 2876, 2879, 2882, 2894-2899, 2969-2995, 3016-3028, 3033-3060, 3064, 3067-3069, 3072-3079, 3083-3104, 3108-3127, 3130-3150, 3153-3154, 3161-3175, 3178-3179, 3183-3196, 3205-3230, 3241-3243, 3246-3248, 3251-3261, 3264-3269, 3272-3274, 3277-3278, 3283-3284, 3287, 3290, 3293, 3303-3308, 3312-3313, 3322-3331, 3335-3350, 3354, 3366-3373, 3377, 3384-3402, 3406-3411, 3416-3424, 3432-3445, 3453-3469, 3472-3475, 3478-3481, 3484-3487, 3490-3491, 3494-3495, 3502-3511, 3515-3525, 3528-3535, 3542-3543, 3553-3554, 3559-3560, 3567-3574, 3580-3585, 3610-3618, 3621-3622, 3626-3627, 3630-3634, 3661, 3705
ops/pebble.py               1279    890    431     12    26%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 309-314, 318-323, 330-331, 335, 345, 350-352, 359, 385-386, 389, 392, 412-416, 419, 433-434, 437-449, 452, 491-494, 497-506, 528, 533, 536, 551-556, 561, 572, 591-593, 598, 605, 616, 634-642, 647, 661, 678, 697-706, 711, 726, 748-759, 769, 777, 785, 789-795, 799, 804-808, 830-842, 847, 851-859, 862, 866-871, 880-902, 906-930, 938-946, 949, 953-958, 985-987, 991, 996-1004, 1011, 1022-1047, 1051-1063, 1066, 1070-1075, 1097-1106, 1110-1118, 1121, 1124-1129, 1190-1199, 1204-1208, 1222, 1279-1283, 1288-1296, 1305, 1377-1381, 1452-1466, 1469-1471, 1483-1485, 1488-1520, 1534-1561, 1570-1577, 1582-1588, 1597-1611, 1617-1637, 1644, 1648, 1652-1655, 1659, 1666-1668, 1672, 1676-1705, 1709, 1739-1746, 1751-1756, 1772-1781, 1790-1795, 1804-1832, 1836-1837, 1841-1843, 1847-1849, 1855-1859, 1863-1864, 1868-1870, 1887, 1905, 1926, 1947, 1968, 1974-1990, 2015-2019, 2023-2044, 2048-2063, 2068-2077, 2089-2109, 2113-2114, 2122-2126, 2130, 2134, 2156-2194, 2198-2204, 2237-2256, 2264-2275, 2281-2317, 2338-2348, 2375-2384, 2400-2408, 2430, 2452, 2596-2709, 2712-2717, 2720-2722, 2736-2749, 2767-2773, 2790-2800, 2808-2809, 2842-2852, 2859-2882, 2885-2904, 2907-2925, 2929-2931, 2935, 2938-2940, 2943, 2947, 2951, 2955-2962, 2990-3011, 3015-3057, 3067-3092
ops/storage.py               159     98     30      0    32%   44-47, 59-68, 72-84, 90-104, 108, 112, 129-130, 141-146, 153, 157-164, 168, 173, 190-208, 221, 243, 254-258, 265, 269-271, 275-277, 289-293, 301-307, 315, 355-356, 376-382, 393-396, 406, 413, 416
ops/testing.py              1589   1589    768      0     0%   15-3568
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6514   4752   2553     38    23%

Test case num increased because of parametrize.

The other test file test_infra.py has only 1 test case so it's not compared.

@IronCore864 IronCore864 marked this pull request as ready for review April 23, 2024 08:17
tonyandrewmeyer
tonyandrewmeyer reviewed Apr 23, 2024
View reviewed changes
Copy link
Contributor

@tonyandrewmeyer tonyandrewmeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving the parametrizing up makes the tests much nicer, I think. A few small comments/questions.

test/test_lib.py Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
benhoyt
benhoyt reviewed Apr 23, 2024
View reviewed changes
test/test_lib.py Outdated Show resolved Hide resolved
test/test_lib.py Outdated Show resolved Hide resolved
tonyandrewmeyer
tonyandrewmeyer approved these changes Apr 24, 2024
View reviewed changes
Copy link
Contributor

@tonyandrewmeyer tonyandrewmeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@IronCore864 IronCore864 merged commit fe9c563 into canonical:main Apr 24, 2024
26 checks passed
@IronCore864 IronCore864 deleted the pytest-phase2-test-infra-lib branch April 24, 2024 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tonyandrewmeyer tonyandrewmeyer tonyandrewmeyer approved these changes

@benhoyt benhoyt benhoyt approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@IronCore864 @tonyandrewmeyer @benhoyt